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A. Summary of Related, Non-Peer-Reviewed Studies 

Although this brief focuses on the three peer-reviewed studies sponsored by IES, these are just part of 
a growing body of evidence that examines access to effective teaching based on student outcomes rather 
than teacher inputs. Table A. 1 summarizes the key aspects of five studies from this emerging literature. 

The table indicates the number of states and districts in each study, subject areas, grade levels, time periods, 
approaches to measuring effective teaching and access to effective teaching, and a summary of the key 
findings. Findings from these additional studies are consistent with the overall finding presented in the 
brief; disadvantaged students have access to less effective teaching on average. 

B. Methods for Comparing Findings from the Sass et al. and 
Isenberg et al. Studies 

The studies we reviewed in the brief used different methods that had to be reconciled before we could com- 
pare the findings. For example, Isenberg et al. (2013) compared the average teacher value added for FRL 
and non-FRL students. Sass et al. (2012), on the other hand, compared teacher value added for higher- and 
lower-poverty schools without accounting for differences in effective teaching between FRL and non-FRL 
students within these schools. 1 The latter approach may underestimate the differences in effective teach- 
ing if FRL students are assigned to less effective teachers within schools. However, Sass and colleagues 
provided additional detail that we were able to use to account for these differences. 2 

To present the findings from Sass et al. using an analogous metric to that of Isenberg et al., we recalculated 
the results from Sass et al. based on a comparison of average teacher value added for FRL and non-FRL 
students. Besides providing more comparable results, this approach accounts for three sources of unequal 
access: differences between districts, between schools, and within schools. We describe our approach to 
re-calculating the Sass et al. results at the student level below. 

Our approach was to use the estimates of average value added for the four combinations of school and 
student type listed below, estimate their sample proportions, and compute weighted averages by student 
type instead of school type, as Sass et al. had done. The four groups are as follows: 

(1) FRL students in higher-poverty schools 

(2) FRL students in lower-poverty schools 

(3) Non-FRL students in higher-poverty schools 

(4) Non-FRL students in lower-poverty schools 
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Table A.1. Key Aspects of Non-Peer-Reviewed Studies on Access to Effective Teaching 


Aspect of the 
Study 

Tennessee 
Department of 
Education (2007) 

Steele et al. 
(2010) 

Mansfield 

(2010) 

Hahnel and 
Jackson (2012) 

Students First 
(2013) 

Number of states 
and districts 

All districts in 
1 state (Tennessee) 

1 district (large, 
urban district in 
the South) 

All districts in 
1 state 

(North Carolina) 

1 district 

(Los Angeles, CA) 

1 district 
(New York, NY) 

Subjects 

Math 

Math, English/ 
language arts, 
science, and 
social studies 

Math, English/ 
language arts, 
science, and 
social studies 

Math and English/ 
language arts 

All subjects 

Grade levels 

4 through 8 a 

4 through 8 

9 through 12 

3 through 1 1 

(English/language 

arts) 

3 through 8 (math) 

Elementary, 
middle, and high 
school grades 13 

Time period 

2005-06 

2004-05 through 
2008-09 

1997-98 through 
2005-06 

2007-08 through 
2009-1 0 

2011-12 

Measure of 
effective teaching 

Value-added score 

Value-added score 

Value-added score 

Value-added score 

District’s teacher 
evaluation rating 

Measure of access 
to effective 
teaching 

Compare the 
percentage of 
highest- and 
lowest-performing 
teachers in schools 
with the highest 
and lowest rates of 
FRL and minority 
students 

Compare average 
teacher effective- 
ness for school 
quartiles based 
on student 
performance 
and proportion of 
minority students 

Compare average 
teacher effective- 
ness for highest 
and lowest quartile 
schools based 
on student 
performance 
and proportion of 
FRL and minority 
students 

Compare percent- 
age of FRL and 
non-FRL students 
who were taught 
by teachers in 
the top and 
bottom quartile 
for effectiveness 

Compare percent- 
age of teachers 
receiving an unsat- 
isfactory evaluation 
rating in schools 
with the highest 
and lowest student 
achievement as 
well as proportion 
of FRL and 
minority students 

Main finding, 
according to the 
authors 

The highest- 
performing teachers 
were under- 
represented in 
schools with the 
highest proportion 
of FRL and 
minority students 

Schools with 
lower-achieving 
students and more 
minority students 
received less 
effective teaching 
in math, reading, 
science, and social 
studies than 
other schools 

Schools with the 
lowest-achieving 
students as well 
as schools with 
the most FRL and 
minority students 
received less 
effective teaching 
than other schools 

Disadvantaged 
students were less 
likely to be taught 
by the highest- 
performing teachers 
and more likely to 
be taught by the 
lowest-performing 
teachers 

A higher percent- 
age of teachers 
received an unsat- 
isfactory rating 

(1) in schools with 
a high proportion 
of FRL and minority 
students and 

(2) in low-performing 
schools 


a. There is no mention of the grade levels in this study. Given that the state assessment covers grades 3 through 8, we assumed the study 
focused on grades 4 through 8. Third grade would not be included because the value-added analysis requires one year of test scores to 
serve as a pre-test. 

b. The authors of this study do not define the grade levels included, but the study includes elementary, middle, and high schools. 
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Whereas Sass et al. reported the weighted average value added of groups (1) and (3) together and com- 
pared it to the weighted average of (2) and (4) together, we calculated the weighted average of (1) and (2) 
together and compared it to the weighted average of (3) and (4), to be comparable to Isenberg et al. 

To obtain the weights, we first calculated the proportion of all FRL students in each state who were in 
lower-poverty schools, with the remainder being in higher-poverty schools. We used information reported 
in Sass et al. on (1) the proportion of teachers in each school type (high versus low poverty) and (2) the 
proportion of students in each school type who were FRL eligible (Table B.l). We assumed that the 
proportion of all students in each school type was equal to the proportion of all teachers in each school 
type. These weights are shown in Table B.l, columns (c) and (d). Weights in column (c) were used with 
value-added estimates in column (e) and weights in column (d) were used with value-added estimates in 
column (f) to produce average value added by student type. Table B.2 shows the same weights applied to 
value-added estimates for English/language arts. 


Table B.l. Calculating Differences in Effective Teaching Between Non-FRL and FRL Students for 
Math Using Data from Sass et al. (2012) 



Estimated Sample 
Proportions (weights) 

Average Teacher Value Added in Standard 
Deviations of Student Achievement 


(a) 

Proportion of 
All Teachers 

(b) 

Proportion of 
Students 
Eligible for 
FRL 

(c) 

Non-FRL 
(a)*[1 -(b)] 

(d) 

FRL (a)*(b) 

(e) 

Non-FRL 

Students 

(f) 

FRL 

Students 

(g) 

Difference 
Between 
Non-FRL and 
FRL Students 

Florida 








Lower-poverty 

schools 

0.65 

0.40 

0.390 

0.260 

0.0439 

0.0242 


Higher-poverty 

schools 

0.35 

0.85 

0.053 

0.298 

0.0349 

0.0240 


Weighted 

average 





0.0429 

0.0241 

0.0188* 

North Carolina 








Lower-poverty 

schools 

0.81 

0.41 

0.478 

0.332 

0.0330 

0.0026 


Higher-poverty 

schools 

0.19 

0.86 

0.027 

0.163 

-0.0100 

-0.0216 


Weighted 

average 





0.0307 

-0.0054 

0.0361* 


Source: Sass et al. (2012) with our calculations in bold. 

Note: Higher-poverty schools are those with 70 percent or more of their students eligible for FRL. The remainder are lower-poverty schools. 
The average teacher value-added estimates for FRL and non-FRL students in each school type are from Tables 4a and 4b in Sass et al. (2012). 
Weighted averages in column (e) use weights from column (c). Weighted averages in column (f) use weights from column (d). 

* Difference between non-FRL and FRL students is significantly different from zero at the 0.05 significance level. 
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Table B.2. Calculating Differences in Effective Teaching Between Non-FRL and FRL Students in 
Reading Using Data from Sass et al. (2012) 



Estimated Sample 
Proportions (weights) 

Average Teacher Value Added in Standard 
Deviations of Student Achievement 


(a) 

Proportion of 
All Teachers 

(b) 

Proportion 
of Students 
Eligible for 
FRL 

(c) 

Non-FRL 
(a)*[1 -(b)] 

(d) 

FRL (a)*(b) 

(e) 

Non-FRL 

Students 

(0 

FRL 

Students 

(g) 

Difference 
Between 
Non-FRL and 
FRL Students 

Florida 








Lower-poverty 

schools 

0.65 

0.40 

0.390 

0.260 

0.0482 

0.0268 


Higher-poverty 

schools 

0.35 

0.85 

0.053 

0.298 

-0.0018 

-0.0229 


Weighted 

average 





0.0424 

0.0003 

0.0421* 

North Carolina 








Lower-poverty 

schools 

0.81 

0.41 

0.478 

0.332 

0.0307 

0.0070 


Higher-poverty 

schools 

0.19 

0.86 

0.027 

0.163 

-0.0335 

-0.0450 


Weighted 

average 





0.0273 

-0.0101 

0.0374* 


Source: Sass et al. (2012) with our calculations in bold. 

Note: Higher-poverty schools are those with 70 percent or more of their students eligible for FRL; the remainder are lower-poverty schools. 
The average teacher value-added estimates for FRL and non-FRL students in each school type are from Tables 4a and 4b in Sass et al. (2012). 
Weighted averages in column (e) use weights from column (c). Weighted averages in column (f) use weights from column (d). 

* Difference between non-FRL and FRL students is significantly different from zero at the 0.05 level. 


We tested the statistical significance of the difference in average teacher value added for FRL and non-FRL 
students by using additional information provided by the authors on the standard errors of average teacher 
value added for teachers of FRL and non-FRL students within each school type (Table B.3). 

We used the data in Table B.3 to calculate the variance of the difference in average teacher value added 
between FRL and non-FRL students for Sass et al. Equation 1 shows this variance expressed as the variance 
of average teacher value added for FRL students [Var(VA FRL )J plus the variance of average teacher value 
added for non-FRL students [Var(VA nonFRL )], minus two times the covariance of average value added for 
the two types of students. However, Sass et al. provided information on the variance of average teacher 
value added separately for teachers in higher- and lower-poverty schools (as shown in Table B.2). Equations 
2 and 3 show how we defined the variance in average teacher value added for FRL and for non-FRL 
students when using these separate variance estimates for higher- and lower-poverty schools (for example, 
VA nonFRL Hp is the variance in average teacher value added for non-FRL students in higher-poverty schools). 
Equation 4 defines the covariance of the average value added for FRL and non-FRL students using separate 
variance estimates for higher- and lower-poverty schools. Equation 5 combines equations 2, 3, and 4. 3 
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Table B.3. Standard Errors of Average Teacher Value Added for FRL and Non-FRL Students in 
Higher- and Lower-Poverty Schools 



Math 

Reading 


FRL Students 

Non-FRL Students 

FRL Students 

Non-FRL Students 

Florida 






Lower-poverty 

schools 

0.0004 

0.0002 

0.0002 


0.0002 

Higher-poverty 

schools 

0.0005 

0.0012 

0.0004 


0.0008 

North Carolina 






Lower-poverty 

schools 

0.0005 

0.0004 

0.0004 


0.0003 

Higher-poverty 

schools 

0.0009 

0.0019 

0.0007 


0.0015 


Source: Information provided by the authors of Sass et al. (2012). 


We then calculated the standard error based on the variance in equation 5 and tested the statistical significance 
of differences in average teacher value added between non-FRL and FRL students. 


( 1 ) 

( 2 ) 

(3) 

(4) 

(5) 


Var{VA mnFSL -VA FRL ) = Var{VA m 


, ) + Var(VA FRL ) - 2 x Cov(L4 


VartVA ) = Var\(VA x Pet ) + (VA x Pet 

V nonFRL ' nonFRL , HP nonFRL, HP ’ V nonFRL,LP nonFRL, LP 

= (Pet ) 2 x VartVA ) + (Pct ) 2 x VartVA ) + 0 

nonFRL, HP > \ r nonFRL , HP ' ' nonFRL, LP ' ^ nonFRL, LP ' 



VartVA ) = Var\(VA xPct ) + (VA x Pet )1 

V - FRL ' ' w LV r FRL, HP FRL, HP' ' FRL,LP FRL.LP'l 

= (Pet ) 2 x Var(VA ) + (Pct f xVar(VA ) + 0 

V FRL,HP J x FRL, HP ' V FRL,LP ' ' FRL,LP ' 


C ° V ( VA „„„FRL’ VA fJ = C ° V {l( VA m FRL,HP XPC L n FRL.Hp) + ( VA o,,FRL,LP XPC L,,FRL.Lp)U( VA FRL,HP XPC KRL^ 
= 1 Pct «onFRL,HP X Pct FRL.HP X C ° V ( VA o„FRL.HP ’ VA FRL.Hp) 1 + 1 PC KonFRL,LP X Pct FRL,LP X C ° V ( VA onFRL,LP > VA FRL,LP )1 


y^(VA tmaa -VA F J = 

{( p ct m „y x Var(VA mFi!L H „) + (, Pct mnFSL J x Lrr(K4_ „)] + [(P C /,„,„„) 2 x Var(VA FRl HP ) + {Pct FRL LP ) 2 x ip )] - 

[ 2xPc L^,^ xPc W xCo ^A^,^.^, H p)]-[ 2xPc W iP xPc W xCov ( F 4 0 »™, u .>^ J!I , i p)] 


Although Sass et al. presented results separately by state, we combined the results from the two states for 
this brief. For each subject, we averaged the results for the two states as shown in Table B.4. To calculate 
the variance of the difference in average teacher value added between FRL and non-FRL students for both 
states, we summed the variance of the two states and assumed that the covariance between the estimates 
from the two states was zero. 
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Table B.4. Aggregating Differences in Average Teacher Value Added Between FRL and Non-FRL 
Students for North Carolina and Florida 



Differences in Average Teacher Value Added Between Non-FRL and FRL Students 

Reading 


Florida 

0.042 

North Carolina 

0.037 

Two-state average 

0.040 

Math 


Florida 

0.019 

North Carolina 

0.036 

Two-state average 

0.027 


C. Converting Standard Deviations into “Weeks of Learning” 

Both Isenberg et al. (2013) and Sass et al. (2012) presented their results as standard deviations of student 
achievement. To make these results more meaningful for educators, we converted the standard deviations 
into “weeks of learning.” In this section, we discuss the conversion process and the assumptions we made. 

From Hill et al. (2008), we obtained estimates of the number of standard deviations of growth a student 
makes in a year on a typical test. We used these estimates to convert the differences in average teacher 
value added between FRL and non-FRL students from standard deviations to years of learning. We divided 
the differences in value added between FRL and non-FRL students by the average growth that students 
make in a year from Hill et al. (2008). We then converted this from years of learning to weeks of learning 
by assuming a nine-month school year and 4.33 weeks per month. Table C. 1 shows the translation of the 
results from standard deviations of student achievement to weeks of learning. 


Table C.1 . Translating Results from Standard Deviations of Student Achievement to Weeks of Learning 



One Year of 
Achievement Growth 

Difference in Average Teacher Value Added 
Between FRL and Non-FRL Students 


(a) 

(b) 

(c) 


Standard Deviations of 

Standard Deviations of Stu- 

Weeks of Learning, 

Unit 

Student Achievement 

dent Achievement 

=9*4.33*(b)/(a) 

Isenberg et al. (2013), Grades 4-8 




Reading 

0.315 

-0.034 

-4.2 

Math 

0.390 

-0.024 

-2.4 

Sass et al. (2012), Grades 4-5 




Reading 

0.360 

-0.040 

-4.3 

Math 

0.485 

-0.028 

-2.2 


Source: Calculations based on data from Isenberg et al. (2013) and Sass et al. (2012). 

Notes: We assumed a nine-month school year, with 4.33 weeks per month. Hill et al. (2008) provided information on average achievement 
growth for students at each grade level. We averaged these values across grades and weighted by the number of students in each grade for 
Isenberg et al. (2013). Given that we did not have information on the number of students by grade level for Sass et al. (2012), we weighted 
grades equally when taking the average of achievement growth across grades 4 and 5 for that study. 
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Endnotes 

1 Sass et al. (2012) defined higher-poverty schools as those with 70 percent or more of students eligible for FRL. The remainder 
were classified as lower-poverty schools. 

2 Sass et al. (2012) report four versions of their results, based on different value-added models. We focused on the version of the 
value-added model that used the authors’ preferred approach (“partial persistence in prior school inputs and student covariates”). 
However, we used the value-added results where shrinkage had not been applied because that aligns more closely with the 
approach used by Isenberg et al. (2013). 

3 We assumed that the covariance in average teacher value added for higher- and lower-poverty schools is zero because the 
teachers in these two types of schools are independent. 
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